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Naturalistic Inquiry in Educational Technology 



Rhonda S. Robinson 
Lina D. Ong 
Dept. of L.E. P. S. /Instructional Technology 
Northern Illinois University 
DeKalb, Illinois 

Abstract 

The field of educational technology developed in the 1940 's out of several 
theoretical constructs, including perception, communication, and the psychology 
of learning. The basis of the primary research in educational technology was a 
combination of behavioral psychology and systems theory. Because of this early 
orientation, most research in educational technology from 1940-1980 followed 
experimental paradigms which attempted to show causality and/or relationships in 
testing the effectiveness of visual media for instruction. 

More recently, researchers and scholars in educational technology have 
recommended widening the field to include more diversity of method and topics 
for study. Current researchers and educational technology doctoral students are 
being encouraged to investigate new questions and to consider various 
alternative methods of investigation. Ethnographic, descriptive, historical, 
and analytic methods are all being recommended to help answer questions 
involving how any given medium is structured to make meaning, how a medium makes 
its impact, and what the scope of the impact has been. 

This study explores some current ateas of research in educational 
technology and defines the non-experimental research methods which can 
appropriately be used to investigate these topics. If new questions are to be 
researched in non-traditional ways, scholars need guidance and assistance to 
create a new thrust of programmatic research in educational technology. The 
proposed continuation of this study could result in a reference guide which 
compiles research topics/ methods, and sample studies to provide the guidance 
needed . 
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Naturalistic Inquiry in Educational Technology 



Rhonda S, Robinson 
Lina D. Ong 
Dept. of L.E. P. S. /Instructional Technology 
Northern Illinois University 
DeKalb, Illinois 

Introduction 

Emerging as a separate field in the 1940 's, Educational Technology drew 
upon theory bases from psychology, learning, and perception. The field was 
derived primarily from behavioral and cognitive psychology, and consequently 
based its seminal research on strict experimental models appropriate to the 
early questions and hypotheses developed. Media was tested experimentally, and 
found (at the time) to be effective. 

In the forty years of research that followed, the same experimental 
paradigm predominated. New media were tested against old, characteristics of 
learners and specific media were compared, and relationships between learners 
and media were explored, all using various accepted experimental designs. 

More recently, the field has broadened its definition to include 
instructional design, media analysis, and learner attitudes among other topics. 
Researchers are asking a variety of new questions, many of which would be 
difficult to examine using traditional experimental methods. 

Background and Rationale 

Currently, most of the published scholarship in educational technology has 
been based uyon experimental and descriptive studies. The leading researchers, 
those who train future scholars in research methods, have only recently begun to 
accept a full range of research methodologies for educational technology. 
Consequently, educational technology journals publish few studies base! upon 
non-experimental designs. Since reports of alternate methodologies are few, 
researchers in educational technology have only the models of research reports 
in other fields to assist them in research design and reporting. 

The experimental "bias" in educational technology has been questioned by 
many researchers seeking to expand the areas of scholarship in the field. 
Becker (1977) recommended alternate methodologies to approaching educational 
technology research. Cochrane, et al. (1980) suggested that researchers base 
new areas of inquiry on "the ethnography of situations in which people use 
visual materials (an anthropological rpproach)" (p. 247). They stressed the 
importance of recognizing that visual learning is a cultural phenomenon and 
should be studied with techniques and analyses appropriate to cultural 
processes. Heinich (1984) in his N.I.E. funded ten year review paper encouraged 
researchers to engage in more "naturalistic" inquiry. "Through the use of 
naturalistic inquiry, I am sure we will discover important factors ... 
that have been ignored too long ..." (p. 84). Heinich also argued that such 
research should be encouraged in dissertation work and should be more 
disciplined and more perceptive than experimental studies. 
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Alternative methodologies would lead the field of educational technology to 
new questions, and to often ignored areas such as the impact of educational 
technologies on social relationships and educational institutions. Kerr (1985) 
suggested that methods drawn from sociology, policy sciences, and anthropology 
could n shed new light on problems that have traditionally been approached using 
psychological research methods" (p. 4). Kerr felt that asking new questions in 
less traditional ways was critical to the future of education. • 

With such recommendations for alternative methods, why has the field not 
adequately responded? In a symposium presentation at the 1986 Association for 
Educational Communication and Technology Conference, Becker, Heinich, and other 
researchers offered their explanations: Many strong researchers believe 
experimental research is vital and best; most new scholars are encouraged to 
design only experimental studies; alternate methodologies are fairly unfamiliar 
to the field and so are less acceptable to dissertation directors and journal 
editorial boards; and studies which employ case or ethnographic methods are more 
difficult to synthesize into a journal length format. This same panel 
recommended research using philosophical, historical, and developmental inquiry, 
semiotic and content analysis techniques, and several anthropological methods 
including case study and participant observation. 

In summary, psychological, experimentally based research has formed the 
basis of educational technology scholarship. Recent developments in research 
and theory generation have resulted in recommendations for new question areas 
and alternate research methodologies. For several years, such suggestions have 
been proffered, and attempts to follow them have been reported. Symposia 
featuring non-experimental reports have been presented at the Conference of the 
Association for Educational Communications and Technology (Mclsaac, Robinson, 
and Koetting, 1984) and elsewhere, but the methods recommended have not gained 
widespread use or familiarity. 

There are numerous books available which detail research designs and 
statistical procedures. Merrian and Simpson (1984) have added to the usual 
texts by presenting a "full range of methodology for doing research, and by 
supporting points with examples from research studies done with adults in adult 
settings 11 (p. iii). It is just this sort of reference which is required by 
educational technology researchers to encourage and guide them to new 
techniques. The problem areas and theory basis of educational technology needs 
to be expanded, and a reference guide could provide motivation and assistance. 

Purpose 

This project will, when completed, have followed several steps necessary to 
the creation of an alternate research guide for educational technology 
researchers. It will, specifically: 

- review the types of research recommended for future educational technology 
issues and concerns; 

- review the non-exparimental methodologies recommended by many scholars to 
enhance educational technology research; 
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- describe and discuss reported studies which have used non-* ^rimental 
research techniques in educational technology; 

- develop guidelines for theory, issues, and research questions appropriate 
to non-experimental educational technology research; and 

- seek 'out and develop examples or models of studies utilizing alternate 
methods, and discuss their reportage. 

Methodology 

The procedures necessary to the completion of this study include the 
following: 

1. A review of the last decade of instructional technology research would 
concentrate on problem areas, topics, and appropriate questions as well as 
research designs. As noted in the background section, several scholars have 
been reviewing and critiquing the educational technology research literature. 
Direction would be taken from their recommendations for future educational 
technology research. 

2. A survey of materials, texts, compiled readings, etc. used by major 
educational technology academic programs to help students gain research skills 
in their field would be conducted. University programs willing to cooperate in 
such a compilation would be asked to respond to a survey (see Appendix 1). 

3. A review of recommended alternate methodologies, including their 
definitions, advantages and disadvantages, and appropriate implementation. 
Texts such as Isaac and Michael (1981) and Merriam and Simpson (1984) would be 
drawn from and a bibliography for further study would be recommended. 

4. A discussion of the emerging problem areas for educational technology 
research would be" synthesized, and the appropriate methods for their 
investigation would be recommended. 

5. Several (6-8) recent studies would be located as models 1 of reported 
non-experimental educational technology research. 

With these purposes and methods in mind, a report of the first two steps in 
the methodology of this project follows. This is a preliminary report of a work 
in progress, with the expectation that others hearing or reading the report will 
offer ideas, suggestions, argument, or research articles to provide models for 
others. 

Review fif Research, 1976-1986 

An examination of articles published in ECTJ and JID for this period was 
undertaken with three major focus points: topic areas, methodologies, and 
research questions. Also of interest were the discussion sections, especially 
the areas of recommended future research. These focus areas will after further 
review provide the answers to several questions, such as: what has been the 
research in educational technology? What methodologies have been chosen to 
answer what questions? Which topics have attracted the most research, and what 
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areas' have been recommended for more investigation? What conclusions can be 
drawn about the use of experimental and "alternate 11 methods of research in 
educational technology? 

First to emerge were subject areas which allowed the organization of this 
compilation of our recent research history. The following areas emerged, as 
they have been defined and exemplified. 

1. Visual Literacy - includes studies on visual cuing strategies, physiological 
effect of cuing (e.g. scanning behavior, eye movement); various pictorial 
attributes, effects of visualization, and individual differences variables 
(picture-style preferences, picture-interpretation behavior of children). 
Example of study: Responsiveness of Nigerian students to pictorial depth 
cues. 

■ 2. Instructional Development - includes studies which addressed and/or have 
implications for instructional media design/development; studies which 
developed models for message design, presentation, production and 
instructional methods/approaches. 

Example: Supplementing traditional instruction with objective-based 
instructional development . 

3. Computers - CAI, CBI, computers in education and industry. 

Example: Computer-based foreign language instruction: Improving student 
attitudes. 

4. Learning Strategies - studies on mnemonic strategies, imagery, and other 
learning or cognitive strategies such as rote and networking; visual and 
verbal information processing. 

Example: Information processing from screen media: a psycholinguistic 
approach. 

5. Learner Characteristics - types of learners, learner preferences, 
motivation, orientation, abilities, attitudes, etc. 

Example: Interaction between student achievement orientation, locus of 
control, and two methods of college instruction. 

6. Media Utilization/Attributes - media symbol systems, coding, media use in 
education, media selection. ^ 

Example: A survey of school teachers' utilization of media. 

Certainly there may be some other possible topic areas, or alternate 
definitions; however, those chosen were considered valid and helpful for our 
discussion. 

Tables 1, 2, and 3 review the number of articles reviewed, by topic and by 
methodology, (insert tables about here). As shown, the Lost common topics were 
computers in education, instructional development, visual literacy, media 
attributes/utilization, and learning strategies. As expected, experimental 
research was by far the most prevalent, with any of the other methods cited only 
rarely. 
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Of non-experimental methods, the most often used was survey, and surveys 
were most often found necessary in the topic area of computers. Perhaps with 
the introduction of a fairly new technology, surveys were necessary to determine 
use patterns and adoption practices. The topic area that was investigated using 
the widest range of methods was instructional development. While most research 
reports cited experimental method, case/field study, survey, and descriptive 
methods were also used. The delphi technique, naturalistic methods, and 
evaluation studies as well as theory development were also reported. 

Overall, the review of research topics and methodologies provided few 
surprises. Experimental methods were used most often, and were reported in all 
topic areas except mass media and training. "Qualitative" research methods were 
much more rare. Descriptive, historical and philosophic approaches, the more 
commonly accepted methods, accounted for only 12 research reports. Case/field 
study, delphi, and naturalistic methods were cited in only 18 studies. The 
experimental models were obviously found to be the most useful and viable for 
the last 10 years. 

Survey Results 

The survey was introduced at the annual P.I.D.T. conference in Bloomington, 
Indiana, and the mailing list for the meeting was used for initial selection of 
respondents. The A.E.C.T. Human Resources directory was also consulted, as was 
the Educational Media Yearbook. Of the almost 60 university programs offering 
master's and doctoral degrees, 30 were selected for participation in this 
survey. In addition, a letter of intent and the survey were published in the 
R.T.D. Newsletter, with an open request for information from other programs. 

Of this representative group, twenty surveys were returned, 19 from 
university programs and one from a corporate training center. Responses herein 
discussed were from these programs: Arizona State, Brigham Young, Colorado- 
Denver, Florida State, Indiana, Iowa State, Kent State, Manitoba, Minnesota, 
Oklahoma, Ohio State, Pennsylvania State, Texas A & M , and Washington, among 
others . 

The six questions on the survey were intended to elicit information about 
the research course requirements in the program, the research methods taught, 
the books used, the expectations of the department (if any) as to type of 
research encouraged/ and the current topics being researched by faculty and 
students at the institution. A copy of the survey is included as Appendix 1. 
The survey was useful in a number of ways. First, it served to call attention 
to this area of interest, non-experimental research, and discover what interest 
and expertise other faculty have in the area. Second, it provided - a list of 
texts currently used or recommended for teaching research skills. Third, it 
revealed that, while encouraging students to conduct research using "whatever 
method appropriate to their question", most programs responded negatively to 
current research being done using non-experimental methods. There were several 
responses that declared interest in this topic, but very few that displayed much 
expertise or practice with the "alternate" methods of conducting research or 
even posing questions. 

Specifically, then, the survey provided the following responses. All 
programs responding had required research courses for doctoral students, and 
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many had such courses for master's students as well. Most of these courses were 
taught by faculty other than instructional systems faculty/ usually in the 
educational psychology program. Consequently/ respondents 1 familiarity with the 
course content was not as instructor nor designer of these courses. Eight 
programs had their own course or courses in instructional systems research/ 
usually seminars for doctoral students. Texts required for these basic research 
courses included those by Ker linger/ Borg and Gall/ McMillan and Schumacher/ 
Dwyer, Best, and Tuckman. 

Of the programs responding to question 3 W only eight had a separate course 
or unit highlighting non-experimental research methods. Texts used in these 
courses included those by Bogdan and BikleU/ Miles and Haberman/ Spradley/ 
Patton/ Willis, and Guba and Lincoln. These texts consider qualitative research 
in education/ naturalistic inquiry/ and ethnographic methods of research. 

Almost all programs responded that students are encouraged to conduct 
research appropriate to their question/ but ten programs encourage experimental 
methods first/ and seven programs responded that they encourage "alternate" 
methods. Many programs reported that students are advised to seek out courses 
in other university departments which could assist them in gaining skills in 
non-experimental research/ such as anthropology or sociology. Only one program 
had a full course in "alternate" methods/ a seminar at Florida State. Other 
programs such as the one at N.I.U. reported that students in instructional 
systems seminars are provided materials and readings in non-experimental 
research since the required educational psychology research courses teach them 
experimental methods and statistics. 

Several programs listed current research topics being investigated in both 
experimental and non-experimental studies. These included: 



experimental: videodisc instruction 
persuasion and media 
model and theory building 
expert systems 
effects of CAI 
computerized testing 



non-experimental: use of distance technology 

effects of microcomputers on student /interaction 
importance of university film and video collections 
reactions to high school classes taught by satellite 
computers in higher education 
television production 
dissemination of technology 
product development and evaluation/ techniques 
and model development. 

Methodologies for these non-experimental studies included survey, delphi 
technique/ meta-analysis, case study/ participant observation/ literary 
criticism/ film criticism/ interview/ and multiple qualitative methods. The list 
was a substantial enough one to expect that some of these studies would derive 
information reportable at conferences and publishable in future journal volumes. 
From these responses it could be summarized that the interest in "alternate" 
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methodologies is substantial and possibly increasing* Several respondents 
remarked that faculty or students were interested but unfamiliar with such 
research* 11 1 think the more naturalistic approaches are much more suited for 
instructional technology than the harder, quantitative ones/ 1 is a quotation 
that characterizes such interested respondents. Overall, the survey provided 
the expected information: lists of courses, one syllabus, several text 
recommendations, and topic areas currently being investigated both 
experimentally and non-experimentally. Interest in the "alternate" 
methodologies was also evident from the responses* 

With sections one and two of the methodology of this project having been 
discussed, th« question is, what next? Obviously, a more thorough review of the 
currently reported topics and methodologies will reveal the questions and the 
results of the research reviewed (1976-1986). This review will provide a more 
detailed look at the current state of research in instructional technology: the 
purpose, the results and the need for future research. 

Also, this ten year review will summarize the recommendations made in the 
several "review" or "critique" articles which have critically analyzed the 
research over the last ten years and made suggestions for future topic areas 
and methodologies. These articles should prove useful for the development of a 
list of topic areas and emphases considered appropriate for the future. 

In addition to the above review of research, a review of the topics and 
methodologies, purposes, results, and recommendations will be completed for the 
last several years using Dissertation Abstracts and the R.T.D. Proceedings as 
primary sources. When these steps are completed, a fairly total picture of the 
current research should be drawn. Topic areas, methods, questions, results, and 
future recommendations when compiled will provide a clear picture of the current 
state of research in instructional technology. The consideration of future 
topics and methods can be then situated in a full review of the recent past. 

Another, more controversial step also remains in this project. A 
discussion and definition of available "alternate" methcds of research must also 
include the question of the prevailing paradigm under which our research is 
conducted. There is a large difference between recommending surveys or 
interviews or even observation and recommending that the field consider a 
paradigm other than the scientific paradigm now guiding our research. Several, 
presentations at our A.E.C.T. conferences have made clear the need for the field 
to grow in new directions; a growth made possible only by the posing of 
essential questions in new frameworks. A new science model has been 
recommended, as has an artistic model (Eisner) or a semiotic model (Barthes), 
among others. This question of paradigm shift must be discussed and resolved as 
we move into the 1990 's and our research moves into new areas. 

The non-scientific paradigm can be productively used to develop new areas 
and methods of research in instructional technology. Methods such as linguistic 
analysis, phenomenology, case study, grounded theory, action, participation, 
observation, and simulation can all be "borrowed" from other, well-defined and 
developed disciplines and applied to our field. These methods are not 
considered "soft" in other fields, and have as their advantage the development 
of breadth and depth or research not possible in the experimental methods. 
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The survey and research review both showed an increased interest in non- 
experimental research. Less clear is the acceptance of a true paradigm shift in 
instructional technology research* A different type of question could and 
should be posed, according to many in our field. This project proposes, when 
finished, to discuss both the paradigm shift, the "alternate" methods it would 
necessitate, and the various data gathering techniques available to researchers, 
regardless of the paradigm being used. It is hoped that through the completion 
of this project, enough evidence of quality research methodologies, topics, and 
questions will be derived to provide future researchers, and the professors who 
teach them, with helpful guidelines to alternative research in our field. 
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Appendix I 



Rhonda S. Robinson Name 

N.I.U. 

University 

Naturalistic Inquiry in Educational Technology 

Recently, researchers and scholars in educational technology have recommended 
widening the field to include more diversity of method and topics for study. We 
are being encouraged to investigate new questions and to consider various 
alternative methods of investigation. Ethnographic, descriptive, historical and 
analytic methods are all being recommended to help answer questions involving how 
any given medium is structured to make meaning, how a medium makes its impact, and 
what the scope of the impact has been. If new questions are to be researched 'in 
non-traditional ways, scholars need guidance and assistance to create a new thrust 
of programmatic research in educational technology. My work will result in a 
reference guide which compiles research tppics, methods, and sample studies to 
provide the guidance needed. 

This proposed project (full proposal available upon request) requires information 
from the varied programs around the country teaching research skills. Your help 
is greatly appreciated. 

Please answer for your program, to the best of your knowledge. 

1. Is there a separate course or unit which discusses research methods? 
Name of course? Enrollment? 



2. Does this course survey several research methods? 
What text(s) are required? Recommended? 



3. Is a separate course or unit taught which highlights non-experimental 

research methods? Which methods are included? What text(s) are used? Who 
'teaches this course? 



A. Are masters or doctoral students encouraged to investigate questions 
experimentally? Are "alternate" inquiry methods encouraged? . 



5. Have any students recently completed studies which employed alternate research 
methods? Topics explored? Methods chosen? 



6. Please list the research topic(s) or area(s) currently being explored by 
yourself and/or your students. Include research methodology utilized. 



665 is 



As some of you are aware, I am undertaking a study on research 
methodologies in educational technology. My goal is to create a 
reference text which compiles research topics, methods, and sample 
studies for scholars who may want to use non-traditional research 
methods in their investigation of new questions in educational 
technology. In this regard I am conducting a survey of academic 
programs that teach research skills. ■ If you have not already 
participated, please provide the requested information pertaining to 
your program (survey follows). Your help in this survey is highly 
appreciated. 

Please return the survey to me at the address below. Thank you for « 
professional cooperation. 

Sincerely, 



Rhonda S. Robinson 

Associate Professor 

Northern Illinois University 

Dept. of LEPS-Instructional Technology 

DeKalb, IL 60115 




Enclosure 
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